Enhanced storage capacity with errors in scale-free Hopfield neural networks: an analytical study

摘要

The Hopfield model is a pioneering neural network model with associativememory retrieval. The analytical solution of the model in mean field limitrevealed that memories can be retrieved without any error up to a finitestorage capacity of $O(N)$, where $N$ is the system size. Beyond the threshold,they are completely lost. Since the introduction of the Hopfield model, thetheory of neural networks has been further developed toward realistic neuralnetworks using analog neurons, spiking neurons, etc. Nevertheless, thoseadvances are based on fully connected networks, which are inconsistent withrecent experimental discovery that the number of connections of each neuronseems to be heterogeneous, following a heavy-tailed distribution. Motivated bythis observation, we consider the Hopfield model on scale-free networks andobtain a different pattern of associative memory retrieval from that obtainedon the fully connected network: the storage capacity becomes tremendouslyenhanced but with some error in the memory retrieval, which appears as theheterogeneity of the connections is increased. Moreover, the error rates arealso obtained on several real neural networks and are indeed similar to that onscale-free model networks.